Data and storage communication networks are in widespread use. In many data and storage communication networks, data packet switching is employed to route data packets or frames from point to point between source and destination, and network processors are employed to handle transmission of data into and out of data switches.
FIG. 1 is a block diagram illustration of a conventional network processor in which the present invention may be applied. The network processor, which is generally indicated by reference numeral 10, may be constituted by a number of components mounted on a card or “blade”. Within a data communication network, a considerable number of blades containing network processors may be interposed between a data switch and a data network.
The network processor 10 includes data flow chips 12 and 14. The first data flow chip 12 is connected to a data switch 15 (shown in phantom) via first switch ports 16, and is connected to a data network 17 (shown in phantom) via first network ports 18. The first data flow chip 12 is positioned on the ingress side of the switch 15 and handles data frames that are inbound to the switch 15.
The second data flow chip 14 is connected to the switch 15 via second switch ports 20 and is connected to the data network 17 via second network ports 22. The second data flow chip 14 is positioned on the egress side of the switch 15 and handles data frames that are outbound from the switch 15.
As shown in FIG. 1, a first data buffer 24 is coupled to the first data flow chip 12. The first data buffer 24 stores inbound data frames pending transmission of the inbound data frames to the switch 15. A second data buffer 26 is coupled to the second data flow chip 14, and stores outbound data frames pending transmission of the outbound data frames to the data network 17.
The network processor 10 also includes a first processor chip 28 coupled to the first data flow chip 12. The first processor chip 28 supervises operation of the first data flow chip 12 and may include multiple processors. A second processor chip 30 is coupled to the second data flow chip 14, supervises operation of the second data flow chip 14 and may include multiple processors.
A control signal path 32 couples an output terminal of second data flow chip 14 to an input terminal of first data flow chip 12 (e.g., to allow transmission of data frames therebetween).
The network processor 10 further includes a first scheduler chip 34 coupled to the first data flow chip 12. The first scheduler chip 34 manages the sequence in which inbound data frames are transmitted to the switch 15 via first switch ports 16. A first memory 36 such as a fast SRAM is coupled to the first scheduler chip 34 and stores data frame pointers (in the form of flow queues) and flow control information (in the form of flow queue control blocks (“FQCBs”) 37). Flow queues are discussed further below. The first memory 36 may be, for example, a QDR (quad data rate) SRAM.
A second scheduler chip 38 is coupled to the second data flow chip 14. The second scheduler chip 38 manages the sequence in which data frames are output from the second network ports 22 of the second data flow chip 14. Coupled to the second scheduler chip 38 are at least one and possibly two memories (e.g., fast SRAMs 40) for storing data frame pointers and flow control information. The memories 40 may, like the first memory 36, be QDRs. The additional memory 40 on the egress side of the network processor 10 may be needed because of a larger number of flows output through the second network ports 22 than through the first switch ports 16.
FIG. 2 schematically illustrates conventional queuing arrangements that may be provided for a data flow chip/scheduler pair (either the first data flow chip 12 and the first scheduler chip 34 or the second data flow chip 14 and the second scheduler chip 38) of the network processor 10 of FIG. 1. In the particular example illustrated in FIG. 2, the first data flow chip 12 and the first scheduler chip 34 are illustrated, but a very similar queuing arrangement may be provided in connection with the second data flow chip 14 and the second scheduler chip 38. In the queuing arrangement for the first data flow chip 12 and the first scheduler chip 34, incoming data frames (from data network 17) are buffered in the input data buffer 24 associated with the first data flow chip 12 (FIG. 1). Each data frame is associated with a data flow or “flow”. As is familiar to those who are skilled in the art, a “flow” represents a one-way connection between a source and a destination.
Flows with which the incoming data frames are associated are enqueued in (“attached to”) a scheduling queue 42 maintained in the first scheduler chip 34. The scheduling queue 42 defines a sequence in which the flows attached thereto are to be serviced. The particular scheduling queue 42 of interest in connection with the present invention is a weighted fair queue which arbitrates among flows entitled to a “best effort” or “available bandwidth” Quality of Service (QoS).
As shown in FIG. 2, the scheduling queue 42 is associated with a respective output port 44 of the first data flow chip 12. It is to be understood that the output port 44 is one of the first switch ports 16 illustrated in FIG. 1. (However, if the data flow chip/scheduler pair under discussion were the egress side data flow chip 14 and scheduler chip 38, then the output port 44 would be one of the network ports 22.) Although only one scheduling queue 42 and one corresponding output port 44 are shown, it should be understood that in fact there may be plural output ports and corresponding scheduling queues each assigned to a respective port. (However, according to an alternative embodiment, disclosed in above-referenced co-pending patent application Ser. No. 10/015,994, filed Nov. 1, 2001, a group of output ports may be associated with each scheduling queue 42).
Although not indicated in FIG. 2, the first scheduler chip 34 also includes flow scheduling calendars which define output schedules for flows which are entitled to a scheduled QoS with guaranteed bandwidth, thus enjoying higher priority than the flows governed by the scheduling queue 42.
The memory 36 associated with the first scheduler chip 34 holds pointers (“frame pointers”) to locations in the first data buffer 24 corresponding to data frames associated with the flows enqueued in the scheduling queue 42. The frame pointers are listed in flow queues (not separately shown), each of which corresponds to a respective flow that is or may be attached to the scheduling queue 42. The flow queue indicates an order in which frames associated with the flow were received and are to be dispatched.
The memory 36 also stores flow control information, such as information indicative of the QoS to which flows are entitled. The flow control information is stored in flow queue control blocks (“FQCBs”), each of which corresponds to a respective one of the flow queues.
When the scheduling queue 42 indicates that a particular flow attached thereto is the next to be serviced, reference is made to the first frame pointer in the corresponding flow queue in the memory 36 and the corresponding frame data is transferred from the first data buffer 24 to an output queue 46 associated with the output port 44. At the same time, the flow is detached from the scheduling queue 42, and, assuming that at least one more frame pointer remains in the corresponding flow queue, is reattached to the scheduling queue in accordance with a procedure that is described below.
A more detailed representation of the scheduling queue 42 is shown in FIG. 3. As noted above, the scheduling queue 42 is used for weighted fair queuing of flows serviced on a “best effort” basis. In a particular example of a scheduling queue as illustrated in FIG. 3, the scheduling queue 42 has 512 slots (each slot represented by reference numeral 48). Other numbers of slots may be employed. In accordance with conventional practice, flows are enqueued to the scheduling queue 42 in the case of a “reattachment” based on a formula that takes into account both a length of a data frame associated with a flow to be reattached and a weight which corresponds to a QoS to which the flow is entitled.
More specifically, the queue slot in which a flow is placed upon reattachment is calculated according to the formula CP+((WF×FS)/SF), where CP is a pointer (“current pointer”) that indicates a current position (the slot currently being serviced) in the scheduling queue 42; WF is a weighting factor associated with the flow to be enqueued, the weighting factor having been determined on the basis of the QoS to which the flow is entitled; FS is the size of the frame currently being dispatched for the flow to be reattached; and SF is a scaling factor chosen to scale the product (WF×FS) so that the resulting quotient falls within the range defined by the scheduling queue 42. (In accordance with conventional practice, the scaling factor SF is conveniently defined as an integral power of 2—i.e., SF=2n, with n being a positive integer—so that scaling the product (WF×FS) is performed by right shifting.) With this known weighted fair queuing technique, the weighting factors assigned to the various flows in accordance with the QoS assigned to each flow govern how close to the current pointer of the queue each flow is enqueued. In addition, flows which exhibit larger frame sizes are reattached farther from the current pointer of the queue, to prevent such flows from appropriating an undue proportion of the available bandwidth of the queue. Upon reattachment, data that identifies a flow (the “Flow ID”) is stored in the appropriate queue slot 48.
In addition to the “reattachment” situation described above, there are two other cases in which flows are attached to the scheduling queue 42. The first of these two cases is concerned with attachment to the scheduling queue 42 upon arrival of the first frame for a new flow. The second of the two cases is concerned with attachment of a flow to the scheduling queue 42 upon arrival of the first frame after the flow queue for the flow in question has been emptied (i.e., after the last frame pointed to by the flow queue is dispatched). In both of these cases, there is no frame currently being dispatched, and accordingly, there is no size information available for such a currently dispatched frame. It has therefore been proposed in both cases to attach the flow to the scheduling queue 42 at a predetermined fixed distance from the current pointer CP for the scheduling queue 42. However, the present inventors have recognized that this proposed practice may undermine the desired weighted fair queuing in certain situations that may be encountered in the second case, namely attachment of the flow to the scheduling queue 42 after the corresponding flow queue has been emptied. In particular, if a given flow is made up of large but relatively infrequent frames, the predetermined fixed enqueuement distance may be too short to limit the flow in question to the Quality of Service to which it is entitled. Furthermore, where a flow is made up of relatively infrequent short frames, the predetermined fixed enqueuement distance may work to “short change” the flow, i.e., to prevent it from receiving the Quality of Service to which it is entitled.
It is an object of the present invention to assure that a contracted-for QoS is maintained for a flow upon attachment of the flow to a weighted fair queue in a case where a new frame is received for the flow after the corresponding flow queue has emptied.